Sample Size and Tests of Measurement Invariance

Measurement equivalence/invariance (ME/I) can be thought of as operations yielding measures of the same attribute under different conditions (Horn & McArdle, 1992). These different conditions include stability of measurement over time (Golembiewski, Billingsley, & Yeager, 1975), across different populations (e.g., cultures; Riordan & Vandenberg, 1994), different mediums of measurement administration (e.g., web-based survey administration verses paper and pencil measures; Taris, Bok, & Meijer, 1998), or different rater groups in multisource feedback assessments (Facteau & Craig, 2001; Vandenberg, 2002). Under each of these conditions, tests of ME/I are often conducted via confirmatory factor analytic (CFA) methods. These methods have evolved substantially during the past twenty years and are in widespread use (Vandenberg & Lance, 2000). Recently, however, Vandenberg (2002) has made a call for increased research on ME/I analyses and the logic behind them, stating that “A negative aspect of this fervor, however, is characterized by unquestioning faith on the part of some that the technique is correct or valid under all circumstances” (p. 140). Although hundreds of studies using CFA tests of ME/I have been conducted, as of yet researchers have not fully delineated the mechanism via which these tests assess invariance. Using MacCallum, Widaman, Zhang, & Hong’s (1999) seminal study as a guide, this study illustrates the effects of sample size, item communality, and factor over-determination on the power of CFA tests of equality of factor loadings. Moreover, we provide a possible explanatory mechanism for these effects.